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16.  Abstract 

'This  report  contains  the  results  of  an  analysis  of  pilot  response  delays  to  col¬ 
lision  avoidance  commands  displayed  in  the  cockpit  of  a  General  Aviation  Trainer 
(GAT)  simulator.  Data  were  obtained  from  previous  tests  conducted  in  the  Beacon 
Collision  Avoidance/GAT  simulation  at  the  National  Aviation  Facilities  Experimental 
Center  (NAFEC),  Atlantic  City,  New  Jersey.  Subjects  were  general  aviation  pilots 
with  a  wide  range  of  experience.  Statistical  curve  fitting  techniques  are  applied 
to  response  delay  data.  For  fixed  geometries,  velocities,  and  aircraft  response 
rates,  the  separation  at  the.  point  of  closest  approach  between  aircraft  responding 
to  collision  avoidance  system  (CAS)  commands  is  inversely  related  to  the  length  of 
pilot  delay  in  responding  to  the  CAS  command.  In  experimental  simulations  used  to 
measure  that  collision  avoidance  system  effectiveness,  the  modeling  of  pilot 
response  delays  should  be  as  realistic  as  as  is  practicable.  The  results  of  this 
study  provide  a  more  realistic  model  than  previously  available.  The  Gamma  distri¬ 
bution  was  the  best  distribution  applicable  to  the  queuing  processes  which  are  con¬ 
ceptually  similar  to  the  process  that  is  generating  the  pilot  response  delay  times. 
In  terms  of  minimum  error  mean  square,  lack  of  bias,  and  uniformity  of  fit,  the  use 
of  the  Gamma  distribution  was  found  to  be  superior  in  approximating  the  empirical 
data.  The  recommendation  to  use  the  Gamma  distribution  in  modeling  pilot  response 
delays  in  subsequent  experimentation  at  NAFEC  is  made. 
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INTRODUCTION 


PURPOSE. 

This  document  describes  the  results  of  research  conducted  to  investigate  the 
generating  mechanism  behind  the  distribution  of  pilot  response  delays;  specifi¬ 
cally,  delays  in  response  to  beacon  collision  avoidance  system  (BCAS)  commands. 
The  analysis  centers  on  identifying  the  theoretical  distribution  that  most 
closely  models  the  empirical  pilot  response  delay  time  data.  The  investiga¬ 
tion  was  undertaken  at  the  suggestion  of  the  BCAS  Logic  Working  Group.  The 
report  provides  the  necessary  information  to  allow  all  BCAS  logic  fast-time 
and  real-time  studies  to  use  the  same  model  of  the  pilot  response  delays  dur¬ 
ing  research  in  support  of  the  BCAS  effectiveness  analysis. 

BACKGROUND. 

Currently,  several  efforts  are  in  progress  to  analyze  the  effectiveness  of 
various  aspects  of  BCAS  logic.  These  studies  have  been  conducted  using  dif¬ 
ferent  models  for  the  distribution  of  pilot  response  delay  times.  During 
July  -  September  1977,  the  National  Aviation  Facilities  Experimental  Center 
(NAFEC)  conducted  an  experiment  to  measure  pilot  assessment  of  certain  aspects 
of  BCAS  (references  1  and  2).  These  data  were  used  to  examine  the  applica¬ 
bility  of  different  pilot  response  models. 

METHOD. 

The  general  aviation  trainer  (GAT  2A)  is  a  flight  simulator  that  models  a 
medium  piston-powered  twin-engine  aircraft.  Peripheral  equipment  used  to  drive 
the  computer-generated  projection  of  intruders  is  shown  in  figure  1.  Data 
collected  during  the  pilot  GAT  2A/BCAS  interaction  experiment  were  used  as  a 
data  base  for  this  analysis.  Data  were  collected  while  the  pilots  were  flying 
the  GAT  2A  under  simulated  instrument  conditions.  During  these  flights,  visual 
targets  were  presented  along  with  the  BCAS-generated  commands.  Several  types 
of  commands  were  presented;  however,  the  analysis  in  this  report  is  concerned 
only  with  the  delay  in  pilot  responses  to  the  four  positive  BCAS  commands;  turn 
left,  turn  right,  climb,  and  descend.  Figure  2  depicts  the  cockpit  display 
devices  used  to  present  BCAS  conmands  to  the  pilots. 

The  distributions  currently  being  used  to  model  the  pilot  response  delay  to 
BCAS  commands  have  a  6-to-7-second  mean.  The  shapes  of  the  distributions  vary 
widely,  ranging  from  the  Uniform,  with  a  range  of  4.5  seconds  to  7.5  seconds, 
to  a  Normal  (7,  1)  distribution  truncated  at  1  and  9  seconds.  However,  none 
of  these  models  possess  the  heavy  right  tail  that  represents  the  significant 
proportion  of  late  responses  that  occurred.  More  meaningful  results  in 
experimental  simulation  can  be  obtained  if  the  model  of  the  pilot  response  to 
collision  avoidance  commands  included  this  characteristic. 
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This  report  also  presents  the  results  of  an  investigation  of  the  underlying 
generating  mechanism  for  the  pilot  response  delays.  Although  this  study 
specifically  addresses  the  identification  of  a  response  model  associated  with 
pilot  response  delays  to  BCAS-generated  commands,  the  use  of  the  Gamma  distri¬ 
bution  may  be  applicable  to  modeling  other  response  delays.  This  would  be  true 
especially  when  the  generating  mechanism  for  the  response  delays  is  similar 
to  the  generating  mechanism  discussed  in  this  report. 


INITIAL  ANALYSIS 


Initial  analysis  has  been  divided  into  three  sections.  The  data  collection 
technique  and  the  method  of  measuring  response  delays  are  presented.  Basic 
assumptions  are  then  delineated,  followed  by  a  discussion  for  the  necessity  to 
transform  the  raw  data.  The  techniques  used  to  transform  the  data  are  reviewed. 

DATA  SOURCE. 


The  standard  Air  Traffic  Control  Simulation  Facility  (ATCSF)  data  reduction 
and  analysis  (DR&A)  program  (reference  3)  in  conjunction  with  Cal-Comp  plots 
of  all  encounters  was  used  to  identify  the  time  at  which  a  positive  BCAS  com¬ 
mand  was  first  presented  to  a  pilot.  The  second-by-second  printout  from  the 
ATCSF  DR&A  program  provided  the  true  heading  and  altitude  of  the  GAT  2A  dur¬ 
ing  the  time  periods  in  which  positive  commands  were  displayed  to  the  pilot. 
This  second-by-second  sequence  determined  the  time  that  the  GAT  2A  began  its 
maneuver  in  the  commanded  direction.  The  time  span  from  command  presentation 
until  the  GAT  2A  began  to  maneuver  in  the  commanded  direction  is  defined  as 
response  delay  time.  With  the  availability  of  second-by-second  data,  each 
individual  pilot  response  delay  time  could  be  measured  to  the  nearest  second. 
The  amount  of  raw  data  (289  positive  command  periods)  justified  the  use  of  this 
measurement  procedure. 

For  each  positive  BCAS  command,  the  heading  and  altitude  of  the  GAT  2A  during 
the  period  of  command  presentation  was  thoroughly  examined.  Since  the  GAT  2A 
heading  could  be  determined  to  the  nearest  tenth  of  a  degree  and  the  altitude 
to  the  nearest  foot,  the  initiation  of  a  true  pilot  response  could  be  separated 
from  the  second-by-second  random  error  in  the  GAT  2A  heading. 

Positive  commands  could  occur  in  either  of  two  dimensions,  horizontal  or  verti¬ 
cal.  For  cases  where  a  positive  command  occurred  when  an  aircraft  was  already 
maneuvering  in  the  same  dimension  as  the  command  (i.e.,  horizontal  maneuvering 
when  a  turn  command  was  displayed) ,  the  pilot  response  delay  data  were  not  col¬ 
lected.  The  removal  of  these  data  points  helped  eliminate  any  effect  the  air¬ 
craft  aerodynamics  would  have  on  pilot  response  delay  times. 

If  no  pilot  response  was  detected  prior  to  the  termination  of  the  BCAS  com¬ 
mand,  no  pilot  response  delay  data  could  be  collected  for  the  command  period. 
Throughout  the  experimentation,  289  positive  command  periods  occurred.  Using 
the  above  procedures,  264  pilot  response  delays  were  measured. 
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BASIC  ASSUMPTIONS 


Further  investigation  into  the  shape  of  the  distribution  of  the  pilot  response 
delay  times  requires  that  several  nonrestrictive  basic  assumptions  be  made. 
Since  response  data  were  not  collected  when  no  pilot  response  was  detected, 
the  distribution  obtained  is  actually  the  conditional  distribution  of  pilot 
response  delay  times  given  that  the  command  length  equaled  or  exceeded 
the  response  delay. 

The  observed  distributions  of  the  pilot  response  delay  times  were  tabulated 
for  the  eight  combinations  of  the  four  possible  command  directions  and  two 
BCAS  logic  sensitivity  levels  (amount  of  warning  time  provided  to  the  pilot 
before  point  of  closest  approach).  The  differences  in  the  sample  means  were 
tested  using  a  multiple  comparisons  T  test.  No  significant  differences  were 
found.  An  approximate  chi-square  test  did  not  detect  differences  in  the  sample 
variances  of  the  response  delays  due  to  sensitivity  level  or  command  direction. 
This  result  supports  the  assumption  that  the  distribution  of  pilot  response 
delays  are  independent  of  command  direction  and  BCAS  logic  sensitivity.  This 
independence  allowed  the  pooling  of  data  to  form  a  single  data  base.  Results 
of  the  approximate  chi-square  test  and  of  the  pooling  strategy  are  reviewed 
in  appendix  A.  Figure  3  presents  the  histogram  of  the  pooled  raw  data  points. 

DATA  TRANSFORMATION. 


The  raw  data  presented  in  figure  3  required  two  modifications  prior  to  begin¬ 
ning  any  curve-fitting  process.  The  first  modification  involved  the  Identifi¬ 
cation  and  elimination  of  excessively  large  response  delays  from  the  data 
base.  The  second  modification  transformed  the  raw  data  to  eliminate  the  bias 
caused  by  the  command  length  distribution.  Since  data  were  only  collected 
when  the  length  of  the  command  presentation  period  exceeded  the  pilot  response 
delay,  the  resulting  data  base  was  biased  by  the  distribution  of  the  lengths 
of  command  periods. 

LARGE  RESPONSE  DELAYS.  The  original  data  base  contained  observations  which 
ranged  from  1  second  to  48  seconds.  The  collision  avoidance  system  being  tested 
generated  commands  15  to  25  seconds  prior  to  the  closest  point  of  approach. 
Analysis  indicated  that  with  response  delays  of  15  seconds  or  more,  the  achieved 
separations  were  about  the  same  as  what  would  have  resulted  with  no  response 
at  all.  The  original  data  set  of  264  observations  included  11  observations 
where  the  response  delays  were  15  seconds  or  greater.  (P  {delay  ^  15}  =  0.042). 
Instead  of  trying  to  fit  the  11  points,  which  in  essence  represented  no  response, 
the  fit  was  conducted  in  the  range  of  1  to  14  seconds.  The  11  observations  in 
question  represent  the  probability  of  no  response  which  can  be  modeled  separately 
from  the  modeling  of  response  delays. 

UNBIASING  TECHNIQUES. 

->  stated  before,  the  observed  distribution  is  the  conditional  distribution  of 
pilot  response  delays  given  that  the  command  length  is  at  least  as  long  as  the 
response  delay.  The  observed  data  do  not  include  some  responses  that  would 
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FIGURE  3.  HISTOGRAM  OF  RAW  DATA  POINTS 


have  occurred  had  some  BCAS  commands  not  terminated  prior  to  14  seconds  dura¬ 
tion.  Therefore,  some  data  have  been  censored  by  the  sampling  scheme;  i.e., 
the  cases  where  pilot  response  delays  exceeded  the  command  length.  This  fact 
indicates  that  the  observed  distribution  has  been  biased  by  the  censoring  of 
some  response  delays  that  would  have  occurred  had  all  command  lengths  been  at 
least  14  seconds  long.  To  correct  for  the  bias,  let  the  events; 


A  =  the  occurrence  of  a  command  length  equaling  or  exceeding  the 
response  delay 

and 

Bk  =  the  occurrence  of  a  response  delay  of  k  seconds,  k  *  1,2,  ..,  14 


Note  the  sets  Bi,  are  mutually  exclusive. 

p(B  |A) 

P(V  "  P(ApkT- 


Bayes  Formula  (reference  4)  states: 


B.)P(B.) 
J  J 


Now,  P(Bk  I  A)  is  the  probability  of  observing  a  response  delay  of  k  seconds 
given  that  the  command  length  equals  or  exceeds  the  response  delay.  These 
probabilities  are  obtained  directly  from  the  distribution  of  observed 
pilot  response  delays. 


P(A  |  Bk)  is  the  probability  that  the  command  length  equals  or  exceeds  the 
the  response  delay  given  the  response  delay  was  k  seconds.  This  is  the 
unconditioned  probability  that  a  command  length  equals  or  exceeds  k  seconds. 
Since  all  positive  BCAS  command  displays  must  equal  or  exceed  4  seconds, 

P(A  I  Bk)  =  1.0  for  k  =  1,  2,  3,  4. 

Additionally,  the  BCAS  display  is  updated  once  every  2  seconds;  hence,  possible 
commands  lengths  are  4,  6,  8,  10,  12,  14,  .  .  .  seconds. 


This  fact  implies  that 

P(A|Bk_i)  =  P(A  |  Bk)  for  k  -  6,  8,  10,  12,  14. 

Values  for  these  probabilities  were  obtained  from  the  empirical  distribution 
of  command  lengths, 
n 

Although  Z  P(A|Bj)  P(B j)  is  unknown,  that  sum  is  independent  of  k,  and  the 

fact  that  P(Bk)  is  a  probability  distribution  can  be  used  to  obtain  unbiased 
values  of  P(Bk) . 

n  n  n 

since  Ep(B  )  =  £p(A(  B.)P(B.)  £  p(B,  |A)/P(a|b  )  =  1 

k«l  k  j  =  l  J  3  k=l  k  1  k 

n 

then  Z  P(A|B  )P(B  )  =  (Z  P(B J  A)/P(A|B  ) 

j  =  l  J  J  \k=l 


7 


and 


P(Br)  =  {Y,  p(BR|  A)/p(A|Bk>  j  P(Bk|A)/P(A|Bk) 


The  effect  of  the  unbiasing  procedure  on  the  original  data  is  reviewed  in 
table  1. 


The  required  evaluation  of 


£p(b  | a)/p(a|b  ) 
k=l  k 


-1 

(1.0855)  =  0.0912 


)  Y  < 

is  obtained  from  table  1.  The  resulting  unbiased  empirical  data  set  J~i|i=2 
is  used  as  the  data  base  for  all  curve-fitting  procedures  that  follow.  The 
plot  representing  the  distribution  of  the  transformed  data  is  shown  in  figure 
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CURRENT  MODELS 


Two  models  are  currently  being  used  for  the  distribution  of  pilot  response 
delays.  The  distribution  in  use  at  the  ATCSF  at  NAFEC  is  the  truncated  Normal 
with  a  mean  of  7  seconds  and  a  standard  deviation  of  1  second.  Upper  and  lower 
truncation  points  of  1  and  9  seconds  are  used  to  limit  the  range  of  the  dis¬ 
tribution.  Mitre,  Inc.,  is  using  a  Uniform  distribution  with  the  range  4.5, 
7.5.  In  this  section,  the  empirical  data  are  fitted  to  several  members 
of  both  the  Normal  and  Uniform  families  of  distributions. 

THE  NORMAL  FAMILY. 


Three  curves  from  the  Normal  family  were  compared  with  the  empirical  distribu¬ 
tion  of  the  transformed  data.  The  three  curves  compared  are  the  best-fit 
nonlinear  least  squares  Normal  curve,  the  Normal  curve  generated  by  the  sample 
moments  of  the  empirical  data,  and  the  ATCSF  model  of  pilot  response  delay. 

BEST  NONLINEAR  LEAST  SQUARES  NORMAL  FIT.  The  BMD07R  program  (reference  5)  was 

used  to  fit  the  data.  BMD07R  uses  the  Gauss-Newton  iterative  procedure 
to  estimate  the  distribution  parameters.  The  system  converged  quite  rapidly. 
Using  this  program,  a  mean  of  5.18  seconds  and  a  variance  of  3.72  seconds 
squared  were  obtained. 

The  analysis  of  the  fit  to  the  14  data  points  in  table  1  is  good;  however, 
several  problems  are  evident  as  shown  on  figure  5.  Comparing  the  unbiased 
data  sample  mean,  5.57  seconds,  with  the  fitted  mean,  5.18  seconds,  shows 
that  the  fit  underestimates  the  data  mean  by  7  percent  and  the  fitted  variance, 
3.72  seconds  squared,  underestimates  the  sample  variance,  4.80  seconds,  by 
20  percent. 

The  symmetric  property  of  the  Normal  causes  an  increasing  underestimation  of 
the  empirical  data  for  all  response  delays  >8  seconds.  The  normal  distribution 
cannot  fit  the  tails  of  the  unbiased  empirical  distribution.  The  estimated 
mean  square  error  (EMS)  of  the  fit  is  8.4  x  10“5. 
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TABLE  1.  EFFECT  OF  UNBIASING  PROCEDURE 
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0.0040  0.7352  0.0054  0.0050 


0.25 


DELAY 


FIGURE  5.  BEST  NONLINEAR  LEAST  SQUARES  NORMAL  FIT 


MAXIMUM  LIKELIHOOD  NORMAL.  The  maximum  likelihood  estimates  of  y  and  a  2 
were  used  to  make  the  next  comparison  from  the  Normal  family.  It  is  noted 
that  the  maximum  likelihood  estimates  of  V  and  o2  are  the  first  two  sample 
central  moments.  Figure  6  presents  the  comparison  of  the  Normal  (5.57,  4.80) 
with  the  empirical  data.  Again,  the  symmetric  property  of  the  Normal  leads 
to  the  downfall  of  the  fit.  This  curve  consistently  underestimates  the  empi¬ 
rical  distribution  in  the  range  3  to  6.5  seconds  and  then  consistently  over¬ 
estimates  in  the  range  6.5  to  10  seconds.  As  before,  this  normal  distribution 
cannot  fit  the  tails  of  the  empirical  distribution.  The  estimated  mean  square 
error  of  this  fit  was  3.0x10“^.  The  differences  in  the  maximum  likelihood 
estimates  of  the  mean  and  variance,  and  the  nonlinear  least  squares  estimate 
(5.57  versus  5.18  for  the  mean  and  4.80  versus  3.72  for  the  variance)  imply 
that  the  underlying  distribution  of  pilot  response  delays  probably  does  not 
belong  to  the  Normal  family. 

NORMAL  (7.  1)  DISTRIBUTION  TRUNCATED  AT  1  AND  9  SECONDS.  This  is  the  distribu¬ 
tion  modeled  for  intruder  response  during  the  BCAS/GAT  2A  experiment.  Figure  7 
compares  this  curve  with  the  unbiased  data.  Again,  the  model  mean  is  biased  (6 
seconds  versus  5.57  seconds).  Truncation  at  9  seconds  prevents  the  modeling 
of  any  responses  greater  than  9  seconds.  From  the  raw  data,  the  probability 
that  a  response  delay  >9  seconds  =  0.13. 

THE  UNIFORM  FAMILY. 


Three  curves  from  the  Uniform  family  were  compared  with  the  empirical  distri¬ 
bution.  The  Uniform  distribution  has  the  form 

f(x)  =  1  for  a  <  x  <  b 

b-a 

=  0  elsewhere 

where  a,  b  are  any  two  real  numbers  such  that  a  <  b. 

Figure  8  presents  the  comparison  of  the  sample  moments  fit  of  the  Uniform, 
the  maximum  likelihood  fit  of  the  Uniform,  and  the  Uniform  distribution  that 
was  previously  used  to  model  pilot  response  delay  times.  None  of  the  three 
uniform  family  curves  fit  the  unbiased  empirical  data  very  well. 


BEST  NONLINEAR  LEAST  SQUARES  FIT  OF  THE  GAMMA  DISTRIBUTION 


A  universally  good  fit  of  the  unbiased  empirical  data  could  not  be  made  from 
either  the  Normal  family  or  Uniform  family.  Other  attempts  were  made  to 
identify  the  distribution.  Some  very  flexible  families,  such  as  the  three- 
parameter  Weibull  family,  were  fitted  without  much  success.  As  a  result,  the 
empirical  data  were  reanalyzed. 


.95 
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FIGURE  6.  MAXIMUM  LIKELIHOOD  NORMAL  FIT 


For  true  normal  family  data  b^  =  0  and  b2  =  3. 

The  sample  distribution  measures  of  b^  and  b2  indicate  that  the  distribution 
is  positively  skewed  and  considerably  more  peaked  than  a  normal  family  member. 
Following  methods  outlined  in  reference  6,  the  plot  of  bi,  b2  on  the  Pearson 
plane  indicates  that  a  Gamma  or  possibly  log  normal  model  may  fit  the  data. 

Two  properties  of  the  empirical  data  are  apparent  from  figure  4.  The  empirical 
distribution  is  not  symnetric,  and  the  distribution  has  a  heavy  right  tail.  In 
attempting  to  identify  the  distribution  of  pilot  response  delays,  an  impor¬ 
tant  thing  to  remember  is  that  the  distribution  is  a  distribution  of  waiting 
times;  specifically,  the  waiting  time  until  a  response  to  a  command.  These 
facts  indicate  the  distribution  might  be  a  member  of  the  Gamma  family. 
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THE  GAMMA  DISTRIBUTION. 


The  Gamma  distribution  is  a  two-parameter  exponential  family  distribution  with 

f(x)  =  xr  1  e  *  x  for  0  £  x  <  ®  (1) 

T(r) 


where ! 


=  0  elsewhere 


\  >  0  and  is  the  scale  parameter 


r  >0  and  is  the  shape  parameter 


Additionally, 


e*1  dt,  T  (r) 


(r-1) !  for  r«l,  2,  3,  .  .  . 


is  called  the  Gamma  Function. 


If  r  =  1  in  (1),  f(x)=  Xe”^x  is  the  exponential  distribution. 

In  queuing  theory,  this  distribution  plays  a  large  role  in  modeling  service 
distributions  and  waiting  time  distributions. 


GENERATING  MECHANISM. 


Karlin  (reference  7)  presented  a  queuing  theory  model  very  analogous  to  the 
process  that  generates  the  pilot  response  delay  times.  The  pilot  is  the 
single  server  in  the  queuing  system.  During  his  busy  period,  he  is  performing 
certain  flight  tasks,  such  as  maintenance  of  heading,  pitch,  attitude,  and  air¬ 
speed.  The  onset  of  these  tasks  represent  the  "arrivals"  in  the  queuing  system. 

The  single  server,  the  pilot,  in  performance  of  the  flight  tasks  can  be  con¬ 
sidered  to  be  a  very  complex  servo  mechanism.  Performance  (servicing)  of 
flight  tasks  is  accomplished  in  r  different  stages,  with  the  possibility  of 
some  tasks  in  different  service  stages  being  served  simultaneously.  A 
specific  task  may  not  need  to  pass  through  all  r  stages  in  order  to  complete 
service.  The  positive  BCAS  command  is  considered  to  be  a  flight  task  to  be 
performed  in  the  service  queue.  The  positive  BCAS  command  is  not  serviced 
until  the  pilot  starts  to  make  control  movements  that  will  cause  the  GAT  2A  to 
begin  to  maneuver  in  the  desired  direction. 

If  the  times  required  to  service  the  arrivals  in  each  service  stage  are  all 
distributed  exponentially  with  parameter  X,  Karlin  has  shown  that  the  waiting 
time  (pilot  response  delay  time)  for  a  task  to  be  serviced  is  distributed  as  a 
Gamma  distribution  with  parameters  r  and  X  .  In  the  same  reference,  Karlin 
has  extended  the  results  to  include  any  general  distribution  of  arrivals.  If 
the  service  times  for  each  service  stage  are  not  identically  distributed, 

McGill  and  Gibbon  (reference  8)  have  shown  that  the  response  time  distribution 
can  be  approximated  by  the  generalized  Gamma  distribution.  A  schematic  of  the 
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possible  service  stages  that  a  pilot  might  perform  in  responding  to  a  command 
is  presented  in  figure  9. 

FITTING  THE  CURVE. 


Since  both  parameters  of  the  Gamma  distribution  are  nonlinear,  the  BMD07R 
Nonlinear  Least  Squares  program  (reference  5)  was  used  to  provide  estimates 
of  the  least  squares  best-fit  Gamma  distribution.  The  parameters  are  esti¬ 
mated  using  a  stepwise  Gauss-Newton  iterative  procedure.  In  addition  to 
equation  (1),  the  partial  derivatives  with  respect  to  the  parameters  were 
needed.  The  partial  derivatives  are  given  by 


3f (x;\,r)  _  xr“VXx 
3X  *  Hr) 


and 


1 

I 


af(x;x,r) 

dr 


.  r-1  -\x 


r2(r) 


nr) 


Log (X  r ) 


r*(r) 


I 

r 


(2) 


(3) 


The  use  of  the  stepwise  Gauss-Newton  iterative  procedure  requires  the  initial 
estimates  of  the  parameters  be  provided.  To  insure  rapid  convergence  in  the 
procedures,  the  maximum  likelihood  estimates  were  used  as  initial  values.  The 
maximum  likelihood  estimates  r  and  A  are  the  solution  to  the  following  system 
of  equations. 


rn 


L  X.  =  0 

A  i=rx 


n  =  253 


(4) 


and 


nLog (  X  )  +  ^  '  Loe(X.)  _  n  •  T  ( r )  -  0 

1 fTTTN — 


(5) 


i=l 


TTrT 


From  reference  9, 


T(r)  *  =  cexp(  %r)  1  n  (1  +  r/i)exp(-r/i)  I  (6) 

I  i-l  ) 


where  Y  =  Euler's  Constant  ■  .57721.  .  . 
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Since  r  (r)  converges  for  r  <  the  infinite  product  in  (6)  can  be  differen¬ 
tiated.  Repeated  application  of  the  chain  rule  and  term-wise  cancellation 
obtained  the  following  expression: 


I 


r<r) 


Z  — 1 — 

i=l  i2(l+r/i) 


(7) 


Substitution  of  (7)  into  (5)  permitted  (4)  and  (5)  to  be  sol  d  by  the  Newton- 
Raphson  method.  After  six  iterations  the  solution  found  was 

A  =  1.271  and  r  =  7.073. 


In  order  to  evaluate  (1),  (2),  and  (3),  two  approximations  were  necessary. 
In  the  first  approximation, 

lXr)~g(r)  where  g(r)  =  Jlnir-l)  (r-l)T_1exp(-r+l  +  l/10r). 

For  7  <  r  <  8,  the  proportional  error  is  I  r(r)  -  g(r) 

The  second  approximation  used  was  I  g(r) 


3000  l  i 

r  (r)  ~  -y+ 

For  7  <  r  <  8,  the  error  of  this  approximation  is  ^.0006. 
were  obtained  from  reference  9. 


Both  approximations 


In  addition  to  providing  the  initial  values  for  the  nonlinear  least  squares 
fitting  procedure,  the  maximum  likelihood  estimates  were  used  to  form  the  MLE 
Gamma  fit  of  the  empirical  distribution.  The  MLE  Gamma  provided  an  excellent 
fit  of  the  empirical  data. 


ANALYSIS  OF  THE  FIT. 

The  14  transformed  data  points  shown  in  table  1  were  fitted  using  the  modified 
BMD07R  program.  The  iterative  procedure  was  nitialized  with  the  maximum 
likelihood  estimates  A  and  r.  The  fitted  values  for  A  and  r  were  1.263  and 
7.092,  respectively.  The  EMS  of  the  fit  was  7.91xl0~5.  The  BMD07R  program 
also  provided  the  approximate  95-percent  confidence  intervals  for  the  para¬ 
meters,  1.234  <  A  <  1.292  and  7.009  <  r  <  7.176.  The  results  of  the  BMD07R 

program  are  included  as  appendix  B. 

The  mean  of  a  Gamma  is  r/ A  .  For  the  fitted  curve,  r/A  =  5.61  compares 
favorably  with  5.57  the  sample  mean.  The  variance  of  the  fitted  Gamma  is 
r/A2  =  4.45.  This  approximates  the  sample  variance  of  4.80.  In  addition,  the 
sample  coefficient  of  skewness,  bi%  =  0.867,  and  the  coefficient  for  the  fitted 
curve,  0.752,  compare  very  favorably.  Table  2  presents  the  analysis  of  the 
fit  of  the  Gamma  distribution  with  A  =  1.263  and  r  =  7.092.  The  nonlinear 
least  squares  best  fit  Gamma  is  graphically  compared  with  the  empirical  data 
in  figure  10. 
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TABLE  2.  ANALYSIS  OF  GAMMA  FIT 


Delay  Experimental  Predicted 


Seconds 

Proportion 

Proportion 

Error 

1 

0.0073 

0.0017 

0.0056 

2 

0.0437 

0.0334 

0.0103 

3 

0.1056 

0.1116 

-0.0060 

4 

0.1747 

0.1821 

-0.0074 

5 

0.1991 

0.2006 

-0.0015 

6 

0.1912 

0.1724 

0.0188 

7 

0.1217 

0.1246 

-0.0029 

8 

0.0695 

0.0795 

-0.0100 

9 

0.0315 

0.0460 

-0.0145 

10 

0.0225 

0.0248 

-0.0023 

11 

0.0140 

0.0125 

0.0015 

12 

0.0093 

0.0060 

0.0033 

13 

0.0050 

0.0028 

0.0022 

14 

0.0050 

0.0012 

0.0038 

COMPARISONS. 

The  Normal  fits  to  the  empirical  data  were  all  affected  by  the  symmetric  pro¬ 
perty  of  the  Normal  distribution.  The  symmetric  property  caused  a  biased 
estimate  of  the  mean  for  the  best  nonlinear  least  squares  Normal  fit. 
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FIGURE  10.  BEST  NONLINEAR  LEAST  SQUARES  GAMMA  FIT 


* 


The  differences  in  the  estimates  of  the  mean  and  variance  for  the  best  nonlinear 
least  squares  estimates  and  the  maximum  likelihood  estimates  (MLE)  tend  to 
invalidate  the  use  of  a  Normal  distribution  in  modeling  the  pilot  response 
delays.  The  minimum  estimated  mean  square  error  for  any  normal  fit  of  the 
empirical  data  is  8.4x10”^. 

Table  3  compares  the  bias  and  standard  deviation  of  each  model  fitted  with 
the  unbiased  empirical  distribution  data.  Although  the  error  mean  square  for 
the  nonlinear  least  squares  normal  fit  almost  matches  the  nonlinear  least  squares 
Gamma,  its  mean  is  highly  biased  (-0.39  seconds).  It  is  highly  unlikely  that 
this  normal  distribution  (the  one  that  fits  the  data  the  best)  is  the  correct 
model  for  pilot  response  delays.  If  in  fact  the  underlying  distribution  was 
Normal  with  a  mean  of  5.18  seconds,  a  T  test  can  be  made  to  compare  this  mean 
with  the  sampled  mean  of  5.57  seconds. 

Let  H  ;  u  -  5.18 
o  o 

V 

then  _  _ 

T  -  (x  -  V  )  yiT  -  2.83 
o 

From  T  tables  with  252  degrees  of  freedom,  the  probability  that  T  >  2.56 
is  .005. 


CONCLUSIONS 


The  Gamma  distribution  fitted  by  the  nonlinear  least  square  method  provided 
the  smallest  estimated  mean  square  error,  7.091x10“^.  The  mean  r/X  and  the 
variance  r/X^  of  the  fitted  Gamma  closely  approximated  the  sample  mean  and 
variance.  The  selection  of  the  Gamma  as  the  model  for  pilot  response  delays 
is  also  supported  intuitively  based  on  a  comparison  of  the  generating  mechanism 
to  a  queuing  theory  model. 

The  results  of  this  analysis  were  hased  on  data  that  resulted  from  experimenta¬ 
tion  using  general  aviation  pilots  as  subjects.  The  simulator  modeled  a 
medium  twin-engine  piston-powered  aircraft.  The  response  model  for  other  pilot 
groups  flying  different  equipment  might  not  be  the  same. 

Analysis  of  the  effect  that  the  use  of  the  incorrect  pilot  response  model  at 
NAFEC  had  on  previous  BCAS  studies  was  made.  No  major  effect  was  detected. 
However,  the  two  distributions  previously  used  do  not  fit  the  emperlcal  data 
as  well  as  the  Gamma  distribution.  These  distributions  did  not  have  the  flexi¬ 
bility  that  the  Gamma  family  has  to  model  both  the  significant  portion  of  early 
responses  (<4  second  delays)  and  the  late  responses  (>8  second  delays) . 

Figure  11  shows  the  differences  in  the  cumulative  probability  functions  for  the 
unbiased  empirical  data,  the  fitted  Gamma,  the  Uniform  (4.5,  7.5),  and  the 
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FIGURE  11.  CUMULATIVE  PROBABILITY  DISTRIBUTIONS 


truncated  Normal  (7,  1).  The  differences  between  the  empirical  data  and  both 
the  Uniform  and  Normal  are  quite  apparent. 


RECOMMENDATION 


For  the  above  reasons,  where  possible  In  BCAS  simulation  studies,  the  distri¬ 
bution  of  pilot  response  delay  times  should  be  modeled  by  the  Gamma  distribu¬ 
tion  with  X  =  1.263  and  r  =  7.092. 


REFERENCES 


1.  Blllmann,  B.,  Report  of  Findings  in  the  BCAS/GAT  II  Interaction  Experiment, 
TM-14-009,  Computer  Sciences  Corporation,  October  1977. 

2.  Strack,  R.  C.,  Beacon  Collision  Avoidance  System/General  Aviation  Trainer 
Pilot  Reaction  Test,  NAFEC,  FAA-NA-77-73-LR. 

3.  Kikkawa,  S.,  Program  Design  Specification  for  the  BCAS  Simulation.  TM-14- 
004,  Computer  Sciences  Corporation,  August  1977. 

4.  Mood,  A.  M.,  et  al.,  Introduction  to  the  Theory  of  Statistics,  McGraw  Hill 
Book  Co.,  1974. 

5.  Dixson,  W. ,  Editor,  BMP  Biomedical  Computer  Programs,  University  of 
California  Press,  January  1973. 

6.  Hahn,  G.  and  Shapiro,  S.,  Statistical  Models  in  Engineering.  John  Wiley  & 
Sons,  Inc.,  New  York,  New  York,  1967. 

7.  Karlin,  S.,  A  First  Course  in  Stochastic  Processes.  Academic  Press,  1966. 

8.  McGill,  W.  J.  and  Gibbon,  J. ,  The  Generalized  Gamma  Distribution  and 
Reaction  Times.  Journal  of  Mathematical  Psychology,  Volume  2,  1968. 

9.  Spiegel,  M. ,  Advanced  Calculus.  Schaum  Publishing  Company,  December  1962. 


26 


APPENDIX  A  -  POOLING  STRATEGY 


Data  collected  during  experimentation  reflected  pilot  response  delays  to  four 
types  of  commands  (climb,  descend,  turn  left,  and  turn  right)  and  two  BCAS 
sensitivity  levels  (high  sensitivity  or  early  warnings  and  low  sensitivity  or 
late  warnings).  The  sample  pilot  response  delay  population  variances  and  all 
sample  sizes  are  depicted  in  table  A-l. 

TABLE  A-l  -  SAMPLE  VARIANCES 


COMMAND 

TYPE 

SENSITIVITY 

NUMBER 

IN  CELL 

*2 

0 

CLIMB 

HIGH 

42 

3.88 

CLIMB 

LOW 

24 

2.72 

DESCENT 

HIGH 

34 

4.97 

DESCENT 

LOW 

26 

5.76 

RIGHT  TURN 

HIGH 

35 

5.43 

RIGHT  TURN 

LOW 

24 

3.20 

LEFT  TURN 

HIGH 

44 

4.00 

LEFT  TURN 

LOW 

24 

5.24 

Prior  to  beginning  any  curve-fitting  procedures,  tests  were  performed  to 
support  pooling  of  data.  Mood,  et  al.  (reference  4)  described  the  following 
approximate  chi-square  test  for  the  equality  of  several  variances. 

Tf  A  O 

a  j  ~  samPle  population  variance  for  population  j 

n^  *  sample  size  for  population  j  j=l,  2,  ...,k 
J  '  k 

N  •Z  n  . 

and  j=l  J 

A  = 


then  the  statistic  -2  Log  (A)  is  approximately  chi-squared  with  (k-1)  degrees 
of  freedom. 


2  2  2 
If  H  :  is  a  =  a  =  .  .  ,  =  a 
o  1  2  k 

2  2 

and  H  :  is  a  fa.  for  some  i  ^  j 

a  i  j  2 

then  the  p  -  .05  test  would  reject  HQ  when  -2Log  (A)  >  x  ^  ^  (.95). 

From  our  data  -2Log  (A)  ■  3.2  and  -2Log  (A) <  X  ^(7)  ■  14.1.  Hence,  the 
hypothesis  that  the  sample  variances  are  all  equal  cannot  be  rejected,  and 
the  pooling  of  data  is  supported. 

The  pooling  resulted  in  a  sample  variance  estimate  of  4.80  seconds  squared 
and  a  mean  of  5.57  seconds. 
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APPENDIX  B 
BMD  RESULTS 


BMD07R  -  NONLINEAR 

LEAST  SQUARES  - 

REVISED  JAN.  1, 

1973 

HEALTH  SCIENCES  COMPUTING  FACULTY, 

UCLA 

PROBLEM  CODE 

GAMMA 

NUMBER  OF  VARIABLES 

2 

INDEX  OF  DEPENDENT 

VARIABLE 

2 

INDEX  OF  WEIGHTING 

VARIABLE 

0 

NUMBER  OF  CASES 

14 

NUMBER  OF  PARAMETERS 

2 

TOLERANCE 

.00100 

EPSILON 

.00010 

MAXIMUM  NUMBER  OF  ITERATIONS 

100 

MINIMA 

3.00E-01 

1.800E00 

MAXIMA 

5.000E00 

3.000E01 

ITERATION 

EMS 

PARAMETERS 

X 

r 

0 

8.2646E-05 

1.2714 

7.0729 

1 

8.0230E-05 

1.2594 

7.1048 

2 

7.9150E-05 

1.2641 

7.0897 

3 

7.9139E-05 

1.2630 

7.0922 

4 

7.9139E-05 

1.2630 

7.0922 

5 

7.9139E-05 

1.2630 

7.0922 

6 

7.9139E-05 

1.2630 

7.0922 

7 

7.9139E-05 

1.2630 

7.0922 

ASYMPTOTIC  STANDARD  DEVIATIONS  OF  THE  PARAMETERS 

2.9185E-02  8.4037E-02 
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